The future of multimodal corpora

the norms of localization in translating persian multimodal texts: the case of videogame demos

abstract هنجارهای بومی سازی در ترجمه متون چندوجهی فارسی:مورد دموهای بازیهای کامپیوتری چکیده اهداف عمده مطالعه حاضر به سه دسته تقسیم میشوند: 1) بررسی مشکلات احتمالی ترجمه دموهای (فیلمهای) بازیهای کامپیوتری،2) تعیین هنجارهای بومی سازی در ترجمه دموهای (فیلمهای) بازیهای کامپیوتری و 3) تعیین ایدئولوژیهایی که این هنجارها در جامعه نشان میدهند. به این منظور، ابتدا، مجموعه ای ازدموهای (فیلمهای) ب...

Multimodal Corpora : How Should Multimodal Corpora Deal with the Situation ?

The paper introduces the notion of multimodal cluster (MMC). MMC is a multicomponent spoken unit, which includes diads “meaning + gesture”, “meaning + phonetic phenomenon” (double MMC) or triad “meaning + gesture + phonetic phenomenon” (triple MMC). All components of the same MMC are synchronized in the speech, gestural and phonetic components conveying the same idea as the semantic component (...

The Design and Implementation of Multimodal Corpora

On the Annotation of Multimodal Corpora

This paper describes our experimental research on the Estonian multi-modal corpus annotation and the annotation scheme that focuses especially on gestures and facial expressions and their relation to such communicative functions as giving feedback and managing turn-taking. The scheme builds on the previous work on feedback and semiotic types of signs, and has been developed in the Nordic cooper...

The FASil speech and multimodal corpora

In the context of the FASiL project, we have studied natural language interactions in a unimodal (speech only) and multimodal (speech and graphics) interface to a personal information management database. We collected multilingual corpora to investigate these interactions in Portuguese, English and Swedish. The corpora are used to train language models, to update acoustic models, to study seman...